{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Glove to word vectors\n",
    "[Glove](https://nlp.stanford.edu/projects/glove/) is an improved model of [word2vec](https://www.tensorflow.org/tutorials/word2vec) without losing too much context in the corps."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import pickle\n",
    "import numpy as np\n",
    "import matplotlib\n",
    "import matplotlib.pyplot as plt\n",
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "filename = 'glove.6B.50d.txt'\n",
    "\n",
    "glove = {}\n",
    "with open(filename, encoding='UTF-8') as f:\n",
    "    for line in f:\n",
    "        entities = line.split()\n",
    "        word = entities[0]\n",
    "        glove[word] = np.array(entities[1:], dtype=np.float32)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "filename = 'glove.6B.50d.pkl'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "with open(filename, 'wb') as f:\n",
    "    pickle.dump(glove, f)\n",
    "    \n",
    "del glove"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "with open(filename, 'rb') as f:\n",
    "    glove = pickle.load(f)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "400000\n"
     ]
    }
   ],
   "source": [
    "print(len(glove.keys()))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "# Keras's IMDB dataset\n",
    "Note Keras's pre-processing of the words. It already set the start_char, oov_char, and index_from."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Using TensorFlow backend.\n"
     ]
    }
   ],
   "source": [
    "from keras.datasets import imdb"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "(x_train, y_train), (x_test, y_test) = imdb.load_data(path=\"imdb.npz\",\n",
    "                                                      num_words=None,\n",
    "                                                      skip_top=0,\n",
    "                                                      maxlen=None,\n",
    "                                                      seed=113,\n",
    "                                                      start_char=1,\n",
    "                                                      oov_char=2,\n",
    "                                                      index_from=3)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "word_index = imdb.get_word_index()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "word_freq = sorted(word_index, key=word_index.get)\n",
    "idx2word = {word_index[w] : w for w in word_freq}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Peek the IMDB comments\n",
    "\n",
    "Note to subtract the index_from and start_char"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "\"this film was just brilliant casting location scenery story direction everyone's really suited the part they played and you could just imagine being there robert redford's is an amazing actor and now the same being director norman's father came from the same scottish island as myself so i loved the fact there was a real connection with this film the witty remarks throughout the film were great it was just brilliant so much that i bought the film as soon as it was released for retail and would recommend it to everyone to watch and the fly fishing was amazing really cried at the end it was so sad and you know what they say if you cry at a film it must have been good and this definitely was also congratulations to the two little boy's that played the part's of norman and paul they were just brilliant children are often left out of the praising list i think because the stars that play them all grown up are such a big profile for the whole film but these children are amazing and should be praised for what they have done don't you think the whole story was so lovely because it was true and was someone's life after all that was shared with us all\""
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "' '.join([idx2word[i - 3] for i in x_train[0][1:]])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Set the vocab size to 5k most commonly used words"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "vocab_size = 5000\n",
    "\n",
    "train = [np.array([i - 3 if i <= vocab_size - 1 + 3 else vocab_size - 1 for i in cmt[1:]]) for cmt in x_train]\n",
    "test = [np.array([i - 3 if i <= vocab_size - 1 + 3 else vocab_size - 1 for i in cmt[1:]]) for cmt in x_test]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "this film was just brilliant casting location scenery story direction everyone's really suited the part they played and you could just imagine being there robert bergman is an amazing actor and now the same being director bergman father came from the same scottish island as myself so i loved the fact there was a real connection with this film the witty remarks throughout the film were great it was just brilliant so much that i bought the film as soon as it was released for bergman and would recommend it to everyone to watch and the fly bergman was amazing really cried at the end it was so sad and you know what they say if you cry at a film it must have been good and this definitely was also bergman to the two little bergman that played the bergman of norman and paul they were just brilliant children are often left out of the bergman list i think because the stars that play them all grown up are such a big bergman for the whole film but these children are amazing and should be bergman for what they have done don't you think the whole story was so lovely because it was true and was someone's life after all that was bergman with us all\n",
      "--------------\n",
      "how his bergman bergman as both man and ape was outstanding not to mention the scenery of the film christopher bergman was bergman as lord of bergman christopher is the soul to this masterpiece i became so with his performance i could feel my heart bergman the of the movie still moves me to this day his portrayal of john was oscar worthy as he should have been nominated for it\n"
     ]
    }
   ],
   "source": [
    "print(' '.join([idx2word[i] for i in train[0]]))\n",
    "print('--------------')\n",
    "print(' '.join([idx2word[i] for i in test[0]]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Padding each review to 500\n",
    "Keras padding 0 at the beginning."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "seq_len = 500\n",
    "\n",
    "from keras.preprocessing.sequence import pad_sequences\n",
    "train = pad_sequences(train, maxlen=seq_len, value=0)\n",
    "test = pad_sequences(test, maxlen=seq_len, value=0)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(25000, 500)\n"
     ]
    }
   ],
   "source": [
    "print(train.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Create embedding matrix\n",
    "Use the 400k glove vector to create an embedding matrix of IMDB movie review."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [],
   "source": [
    "embedding_matrix = np.zeros((vocab_size, 50))\n",
    "\n",
    "for i in range(1, vocab_size):\n",
    "    w = idx2word[i]\n",
    "    v = glove.get(w)\n",
    "    if v is not None:\n",
    "        embedding_matrix[i] = v"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Keras Embedding Layer\n",
    "Note that we set the embedding layer to untrainable."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from keras.layers import Input, Flatten, Dropout, Dense, Embedding, Conv1D, MaxPooling1D, Activation\n",
    "from keras.models import Model, Sequential\n",
    "from keras.layers.normalization import BatchNormalization\n",
    "from keras.optimizers import Adam\n",
    "from keras.layers.recurrent import LSTM"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [],
   "source": [
    "embedding_layer = Embedding(\n",
    "    vocab_size,\n",
    "    50,\n",
    "    weights = [embedding_matrix],\n",
    "    input_length = seq_len,\n",
    "    trainable=False # Not trainable\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "_________________________________________________________________\n",
      "Layer (type)                 Output Shape              Param #   \n",
      "=================================================================\n",
      "input_1 (InputLayer)         (None, 500)               0         \n",
      "_________________________________________________________________\n",
      "embedding_1 (Embedding)      (None, 500, 50)           250000    \n",
      "_________________________________________________________________\n",
      "dropout_1 (Dropout)          (None, 500, 50)           0         \n",
      "_________________________________________________________________\n",
      "conv1d_1 (Conv1D)            (None, 500, 64)           16064     \n",
      "_________________________________________________________________\n",
      "dropout_2 (Dropout)          (None, 500, 64)           0         \n",
      "_________________________________________________________________\n",
      "max_pooling1d_1 (MaxPooling1 (None, 100, 64)           0         \n",
      "_________________________________________________________________\n",
      "flatten_1 (Flatten)          (None, 6400)              0         \n",
      "_________________________________________________________________\n",
      "dense_1 (Dense)              (None, 100)               640100    \n",
      "_________________________________________________________________\n",
      "dropout_3 (Dropout)          (None, 100)               0         \n",
      "_________________________________________________________________\n",
      "dense_2 (Dense)              (None, 1)                 101       \n",
      "=================================================================\n",
      "Total params: 906,265\n",
      "Trainable params: 656,265\n",
      "Non-trainable params: 250,000\n",
      "_________________________________________________________________\n"
     ]
    }
   ],
   "source": [
    "sequence_input = Input(shape=(seq_len,), dtype=np.int32)\n",
    "embedded_sequences = embedding_layer(sequence_input)\n",
    "x = Dropout(.25)(embedded_sequences)\n",
    "x = Conv1D(64, 5, padding='same', activation='relu')(x)\n",
    "x = Dropout(.25)(x)\n",
    "x = MaxPooling1D(5)(x)\n",
    "x = Flatten()(x)\n",
    "x = Dense(100, activation='relu')(x)\n",
    "x = Dropout(.7)(x)\n",
    "y = Dense(1, activation='sigmoid')(x)\n",
    "\n",
    "model = Model(sequence_input, y)\n",
    "model.summary()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "model.compile(\n",
    "    loss='binary_crossentropy',\n",
    "    optimizer='adam',\n",
    "    metrics=['acc']\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Train on 25000 samples, validate on 25000 samples\n",
      "Epoch 1/10\n",
      "25000/25000 [==============================] - 14s - loss: 0.6505 - acc: 0.6002 - val_loss: 0.4918 - val_acc: 0.7863\n",
      "Epoch 2/10\n",
      "25000/25000 [==============================] - 13s - loss: 0.4890 - acc: 0.7714 - val_loss: 0.4232 - val_acc: 0.8155\n",
      "Epoch 3/10\n",
      "25000/25000 [==============================] - 13s - loss: 0.4470 - acc: 0.7969 - val_loss: 0.4022 - val_acc: 0.8282\n",
      "Epoch 4/10\n",
      "25000/25000 [==============================] - 13s - loss: 0.4271 - acc: 0.8074 - val_loss: 0.3852 - val_acc: 0.8314\n",
      "Epoch 5/10\n",
      "25000/25000 [==============================] - 13s - loss: 0.4128 - acc: 0.8154 - val_loss: 0.3893 - val_acc: 0.8383\n",
      "Epoch 6/10\n",
      "25000/25000 [==============================] - 13s - loss: 0.3958 - acc: 0.8236 - val_loss: 0.3694 - val_acc: 0.8405\n",
      "Epoch 7/10\n",
      "25000/25000 [==============================] - 13s - loss: 0.3829 - acc: 0.8302 - val_loss: 0.3673 - val_acc: 0.8449\n",
      "Epoch 8/10\n",
      "25000/25000 [==============================] - 13s - loss: 0.3726 - acc: 0.8352 - val_loss: 0.3694 - val_acc: 0.8426\n",
      "Epoch 9/10\n",
      "25000/25000 [==============================] - 13s - loss: 0.3610 - acc: 0.8419 - val_loss: 0.3618 - val_acc: 0.8422\n",
      "Epoch 10/10\n",
      "25000/25000 [==============================] - 13s - loss: 0.3519 - acc: 0.8464 - val_loss: 0.3581 - val_acc: 0.8454\n"
     ]
    }
   ],
   "source": [
    "history = model.fit(train, y_train, epochs=10, validation_data=(test, y_test))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYwAAAEWCAYAAAB1xKBvAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XucXWV97/HPb+73+0ySmck9ECABAgxJVKogakFEtFTB\nW6ttpShU8GWrtLXVVk+PPdoeTxVFpIitXFQEiRZBUMAbmZAoQhJCSGZymZkkM5k991vm8jt/rDWT\nnWGS7CSzs/fs+b5fr/2avddl799syPrOWs96nsfcHRERkeNJS3QBIiIyMygwREQkJgoMERGJiQJD\nRERiosAQEZGYKDBERCQmCgwRwMzuMbPPx7jtLjN7U7xrEkk2CgwREYmJAkMkhZhZRqJrkNSlwJAZ\nI7wU9Ddm9oKZ9ZnZf5rZHDP7iZn1mNmTZlYatf3bzWyLmXWa2dNmdnbUugvM7Lfhft8FciZ91tvM\n7Plw39+Y2Xkx1niVmf3OzLrNbK+ZfXbS+kvC9+sM138wXJ5rZv9mZrvNrMvMfhUuu9TMmqb4Ht4U\nPv+smT1oZt8xs27gg2a22syeDT9jn5l91cyyovZfYWZPmFnEzA6Y2d+Z2Vwz6zez8qjtLjSzNjPL\njOV3l9SnwJCZ5lrgzcCZwNXAT4C/AyoJ/n/+GICZnQncD9warnsU+JGZZYUHzx8C/w2UAd8P35dw\n3wuAu4G/BMqBbwDrzCw7hvr6gD8BSoCrgI+Y2TvC910Y1vuVsKZVwPPhfl8CLgJeG9b0SWAsxu/k\nGuDB8DPvBUaBjwMVwGuAy4GPhjUUAk8CjwHVwDLgZ+6+H3gaeHfU+34AeMDdh2OsQ1KcAkNmmq+4\n+wF3bwZ+CdS7++/cfRB4GLgg3O464H/c/YnwgPclIJfggLwWyAS+7O7D7v4g8FzUZ9wAfMPd6919\n1N2/DQyF+x2Tuz/t7i+6+5i7v0AQWm8IV78XeNLd7w8/t93dnzezNODPgFvcvTn8zN+4+1CM38mz\n7v7D8DMH3H2Tu6939xF330UQeOM1vA3Y7+7/5u6D7t7j7vXhum8D7wcws3TgPQShKgIoMGTmORD1\nfGCK1wXh82pg9/gKdx8D9gI14bpmP3Lkzd1RzxcCnwgv6XSaWScwP9zvmMxsjZk9FV7K6QJuJPhL\nn/A9dk6xWwXBJbGp1sVi76QazjSzH5vZ/vAy1b/EUAPAI8A5ZraY4Cyuy903nGRNkoIUGJKqWggO\n/ACYmREcLJuBfUBNuGzcgqjne4H/5e4lUY88d78/hs+9D1gHzHf3YuAOYPxz9gJLp9jnIDB4lHV9\nQF7U75FOcDkr2uQhp78ObAPOcPcigkt20TUsmarw8CztewRnGR9AZxcyiQJDUtX3gKvM7PKw0fYT\nBJeVfgM8C4wAHzOzTDP7I2B11L7fBG4MzxbMzPLDxuzCGD63EIi4+6CZrSa4DDXuXuBNZvZuM8sw\ns3IzWxWe/dwN/LuZVZtZupm9Jmwz2Q7khJ+fCXwaOF5bSiHQDfSa2VnAR6LW/RiYZ2a3mlm2mRWa\n2Zqo9f8FfBB4OwoMmUSBISnJ3V8m+Ev5KwR/wV8NXO3uh9z9EPBHBAfGCEF7x0NR+24EPgx8FegA\ndoTbxuKjwD+bWQ/wjwTBNf6+e4C3EoRXhKDB+/xw9V8DLxK0pUSAfwXS3L0rfM+7CM6O+oAj7pqa\nwl8TBFUPQfh9N6qGHoLLTVcD+4FXgMui1v+aoLH9t+4efZlOBNMESiISzcx+Dtzn7ncluhZJLgoM\nEZlgZhcDTxC0wfQkuh5JLrokJSIAmNm3Cfpo3KqwkKnoDENERGKiMwwREYlJSg1UVlFR4YsWLUp0\nGSIiM8amTZsOuvvkvj1TSqnAWLRoERs3bkx0GSIiM4aZxXz7tC5JiYhITBQYIiISk7gGhpldYWYv\nm9kOM7ttivXFZvYjM/t9OG/Bh6LW7TKzF8M5CXSdSUQkweLWhhEOknY7wTAETcBzZrbO3bdGbXYT\nsNXdrzazSuBlM7s3HLoB4DJ3P3gqdQwPD9PU1MTg4OCpvE3Sy8nJoba2lsxMzXUjIvERz0bv1cAO\nd28AMLMHCCZ6iQ4MBwrDUUMLCMbQGZnOIpqamigsLGTRokUcOThp6nB32tvbaWpqYvHixYkuR0RS\nVDwvSdVw5Dj9TeGyaF8FziYYivpFgglkxmcZc+BJM9tkZjcc7UPM7AYz22hmG9va2l61fnBwkPLy\n8pQNCwAzo7y8POXPokQksRLd6P2HBCN2VhNMV/lVMysK113i7quAK4GbzOz1U72Bu9/p7nXuXldZ\nOfWtxKkcFuNmw+8oIokVz0tSzQQT1oyrDZdF+xDwhXDmsx1m1gicBWwIp+DE3VvN7GGCS1y/iGO9\nIiJJb2hklNbuIfZ3D7K/a5AD3YMMjzofuXSq+bemVzwD4zngjHC6x2bgeo6cTAZgD8EE9b80sznA\ncqDBzPIJ5gLoCZ+/BfjnONYaN52dndx333189KMfPaH93vrWt3LfffdRUlISp8pEYjQ2Cn0HoacF\nelshLR2yiyC7MHwUQVYBpCX6gsXM5u509g8HQdA9yIGuQdo6u+jtaGWgs5XhnoN4XzsZhzopo4dS\nCx5n0oNlZMGlP497jXELDHcfMbObgceBdOBud99iZjeG6+8APgfcY2YvEkwh+Sl3P2hmS4CHw8ss\nGQRj8z8Wr1rjqbOzk6997WuvCoyRkREyMo7+9T/66KPxLk1mO3cY6obufdAT9Tji9f7g4aPHf7+s\nwqgQiXrkFE0KmKigmep5Wnr8f/fT7NDIGAcinUTa9tPZvp/ejlaGulsZ6WnH+9tJH+wg+1AHxd5N\nifVyhvWwmh7ybejVbxbeCDmSWchYbhlp+RWkl85/9XZxENehQdz9UeDRScvuiHreQnD2MHm/Bg7P\nRDaj3XbbbezcuZNVq1aRmZlJTk4OpaWlbNu2je3bt/OOd7yDvXv3Mjg4yC233MINNwTt++PDnPT2\n9nLllVdyySWX8Jvf/IaamhoeeeQRcnNzE/ybSVIbGTp8wO9uCQ/84c/oQBjuf/W+OSVQOA+K5kHl\nWcHzwrlQVA0Fc4IzjqGeIGyGeiY9uo9c3rPvyPWvmn58Cpn5Rwme4sPPswogLQMsLephwc+09EnL\n08DSD68ff7xqOwu3m7zv+Lb2qvf0sRH6Otvoat9PX2crQ91tjPS0YwNBCOQMd5I/2kWxdzPfhjja\nYb0/LZ/B3FJGsksgdwHpBeWMFVUwUlxFRkEF5JVDXln4sxxyS8lIP/230KfUWFLH808/2sLWlu5p\nfc9zqov4zNUrjrr+C1/4Aps3b+b555/n6aef5qqrrmLz5s0Tt7/efffdlJWVMTAwwMUXX8y1115L\neXn5Ee/xyiuvcP/99/PNb36Td7/73fzgBz/g/e9//7T+HjJDjI1BX9uxzwi6W2Ag8up907ODECis\nhnnnw/IrgyAonHc4IArmQlZe/Gof7oPB7kkBc5zQGeoJfufo7Sdupkys8f4ABVHLuj2PbiukL72Y\n/qwyenKWsT+3jIzCCrKLKikoraKobA4FpXOwMAjy0jOJ07c+rWZVYCSD1atXH9FX4j/+4z94+OGH\nAdi7dy+vvPLKqwJj8eLFrFq1CoCLLrqIXbt2nbZ6ZRqNjcGh3uAx+SA5sawbhnonLYs6aPYegLFJ\nXZUsDfKrgoN/yQKYvzoIhcK5YUCEj9zS4K/kRElLO3yGcCrcgzOjsdEgOHwsWObRr6MeY6Ph+ujl\nk7cN1vcODtEc6aOlo4+Wzn72dfSxr7Of9t4BzJ00xkg3pyo/k3nF2VQV5ZJbXEFB2RxKyuZSXjmH\nqtJCajNS77IazLLAONaZwOmSn58/8fzpp5/mySef5NlnnyUvL49LL710yr4U2dnZE8/T09MZGBg4\nLbUKhw9OQz3hgbx76oN5LMsO9cb2melZhy+7jF/bL6iCqnOODIDxs4L8KkifRf+UzSAr//jbHYW7\nc6B7iB2tvexo7WFHW2/4vI+DveNtBjlkZeSxpGIhSxcWcH5lAcuqClhaWcCSynxyMlMzEI5nFv1f\nlhiFhYX09Ew922VXVxelpaXk5eWxbds21q9ff5qrkwljY9DdBG0vQ9u28Gf4GOo6/v6WHtV4WxA8\nzysL/uKfvDyr4CjLwtcZ2cf/PDmukdEx9kT6gzAIQ2FnWx8Nrb30DB0+SyvMyWBZVQGXLa+cCIVl\nVQXML8sjPU39m6IpMOKsvLyc173udaxcuZLc3FzmzJkzse6KK67gjjvu4Oyzz2b58uWsXbs2gZXO\nEmOj0LHryGA4+DK0bQ+ur4/LrwwafM/9YyiZ/+q7fCYO+uEjIyexl3tmsYFDo+xs62XnRCgEP3cd\n7OfQ6OG2jjlF2SyrKuCPLqw5IhgqC7PV8TVGKTWnd11dnU+eQOmll17i7LPPTlBFp9ds+l2Pa3QY\nIg1RZwvbglA4uB1Go25VLKyGyuVBOFSeGfysWA755Ud/bzmtxvsnNHcO0NI5QHPnAE0dAxPB0Nw5\nwPhhLM1gYXn+RBgsrcwPflYVUJSjgTmnYmab3L0ulm11hiEz2/AgtO84MhgObg+WRTcOlywIwmDp\npUEgjAdETnHCSpfA8OgYB7oHae4YoKVrgOaOAZo7ByfCoaVzgP5DR/YDyclMY0lFARcuKOXddfMn\nAmJRRR7ZKdrgnAwUGDIzHOoLguCINoZtweWl8VssLQ1KFwdhsPyt4ZnDcqg485QaSeXU9AwO09I5\nSHNn/+Eg6Dh8tnCge5CxSRc6yvOzqCnNZVllAW84s5LqklxqSnKoKcmjuiSHsvwsXUZKAAWGJJZ7\ncAdRfzv0R4L+A/3twaO75XDDc9eew/ukZUD5Mph7Lpz7rjAUlgfLMnMS97vMQmNjTmvP0BGXi8YD\nYfx59+CRtwFnphvzinOpKcnltUsrgiAozQ1DIfg5W+9CSnYKDJk+0Qf/gUgQAONBMB4C/e0w0BH1\nOgJjw1O/X3p2cHYwfzVc+CeHzxjKlkACernOVn1DIzQe7Asblvto6uifCIf9XcHAd9GKczOpLsml\ntjSXNYvLgiCICoTKgmzSdPfRjKTAkKmNH/wHog/6kw/8U4TC0Q7+lga5UUMblC2B2rpwmIOo5RPD\nH5RBdrEGtDtNxvsmNIR3G+1sCwOitZeWrsN9g9IM5hYFZwQXLiidOCOoCUNhXnEOhWpcTlkKDAkM\ndMKOJ2H7Y7Dr10GvYh38U87QyCi72/vZ2frqYOiLalguyM5gaWU+a5eUszS822hJZQELy9WoPJsp\nMJJMQUEBvb0x9gg+Ve07g4B4+Sew59ngrqK8Clh6GRTX6uA/g0X6Dk0EQXQw7I30H9HAXFOSy5LK\nfN5VN5+llcHtqEurCqhS3wSZggJjNhkbhb0bYPtP4OXHgg5rEAw58dqPBYPR1VyUksNLp6KR0TH2\ndgxEhUIvDWEwdPQfPjvMzkhjcUU+K2uKuWZVzUQwLKnMJy9LhwCJnf5vibPbbruN+fPnc9NNNwHw\n2c9+loyMDJ566ik6OjoYHh7m85//PNdcc018Chjshp0/CwLilZ8G7Q5pmbDodVD3Z7D8CihdFJ/P\nlmkxNDLK9v29vNLaE541BKGwq73viAbnioJsllbmc+W584IzhTAYakpy1cgs02J2BcZPboP9L07v\ne849F678wlFXX3fdddx6660TgfG9732Pxx9/nI997GMUFRVx8OBB1q5dy9vf/vbpuwTQsSsIiO0/\nCdojxoaDy0tnvCUIiKWXB5PaSNLpPzTCS/u62dzczZaWLjY3d7P9QA8j4XWkjDRjYXkeSysLeNM5\nc1hSkR+0MVQUUJynxmaJr9kVGAlwwQUX0NraSktLC21tbZSWljJ37lw+/vGP84tf/IK0tDSam5s5\ncOAAc+fOPbkPcQ86tg12wu1roe2lYHnFmfCaj8KZVwa3pupSU1LpGhhmS0sXW1u62dzcxeaWbna2\n9U4Mc1Gen8WKmmIuXV7JyppizpxTyMLyPDLT1X4kiTG7AuMYZwLx9K53vYsHH3yQ/fv3c91113Hv\nvffS1tbGpk2byMzMZNGiRVMOa35M47OeDXYFQ26PjcBgD+RXwIX/AmdeAeXxnxReYnOwd4gtYTCM\nnznsiRye7W5ecQ4rqot523nzWFldzMqaYuYUqeFZksvsCowEue666/jwhz/MwYMHeeaZZ/je975H\nVVUVmZmZPPXUU+zevTu2Nxo5FAy1PdgVzM2AB8Nqj8+Z3JkFH/xxXH8XOTZ3Z3/3IJubjwyH/d2H\n/yBYWJ7HuTXFXL96Piuri1lRXUR5gYY0l+SnwDgNVqxYQU9PDzU1NcybN4/3ve99XH311Zx77rnU\n1dVx1llnTb3j+OQ9g11B4/VIOHFSenYw/HZOUTDM9vhfoXbg9PxCAgThsCfSH4RDS1cYEN1E+g4B\nQSe3pZUFvGZpOSuqi1hRXcw51UUU56qtQWYmBcZp8uKLhxvbKyoqePbZZ6fcrre7K+hENxSGxPiI\nq1kFUFQd9IHQeEmn3eiY09DWGwZDcPawtaV7YiKezHTjzDmFvOnsKlbWFLOiupiz5xXqtlVJKfq/\nOVmMDEJXS9AeMX6pKbswGH47u2h2TcGZYOO3sW5pCc4YtrR0sXVfN4PDwai42RlpnD2viGsuqJ5o\nbzhjToF6QEvK01Eo0dyh/2AwMisWNFrnFAfDcZvuhom3nsFhtrZ0h8EQhMOO1t6J21gLsjM4Z14R\n7129kJU1RaysKWZJRT4ZulNJZqFZERjunpx3m4wegs49wd1O2YXBJD/pWSf1Vqk0c2K8tPUMTZw1\nbA3DYVf74TuVKgqyWVFdxBvPqmJF2Bi9oCxPnd5EQikfGDk5ObS3t1NeXp5coTHQAZ17gzOM4tpg\nDKeTrM/daW9vJydHbRsQfB9NHQMT4TDeGN3ac3hq1vlluayYV8y1F9ayoqaIldXFVBXp+xM5lpQP\njNraWpqammhra0t0KQEfC8LiUF9wt1NeGXQdBA6e0tvm5ORQW1s7PTXOICOjY+xs6zuyvaGle2LS\nnjSDZVUFvG5Zhe5UEjlFKR8YmZmZLF68ONFlBBqegR9+BHr2wxs+CX/wCU0EdAIGh0fZtr8nKhy6\n2bavm6GRw43RZ80t5G3nV0+Ew1lzCzV7m8g0SfnASArDA/Czf4b1X4PyM+AvnghGhZWj6hsa4YWm\nriPOHHa29TEaNkYX5mSworqI969dOBEOSyvVGC0STwqMeGt5Hh66IRhKfPUN8KZ/gqy8RFeVdHqH\nRti4K8L6hgj1je282NQ1cadSVWHQGP2Wc+ZOhMP8stzkapMSmQUUGPEyOgK//r/w9BeCXtnvfwiW\nXZ7oqpJGz+AwG3d1sL6hnfWNETY3dzE65mSkGefVFnPD65dw8aIyVtQUUVWoxmiRZKDAiIf2nfDw\nX0LTc7DyWnjrl4LG7Vmsa2A4PINopz4MiDEPekifX1vCR96wlDVLyrhoYal6R4skqbj+yzSzK4D/\nB6QDd7n7FyatLwa+AywIa/mSu38rln2TkjtsvBt++umgMfva/4Rz/zjRVSVEV/8wGyYCop0tLd24\nQ1Z6Gqvml3DzZctYs6ScCxeUkpulRmmRmSBugWFm6cDtwJuBJuA5M1vn7lujNrsJ2OruV5tZJfCy\nmd0LjMawb3Lp2Q+P3Aw7noAll8E1t0NxTaKrOm06+g5R3xi0P9Q3RHhpfxgQGWlcuKCEj73xDNYu\nKeeCBSW6a0lkhornGcZqYIe7NwCY2QPANUD0Qd+BQgtaLwuACDACrIlh3+Sx5Yfw448Hd0Nd+UW4\n+C8gLbXv1mnvHWJDY4T6xuAsYtv+HiC4tfWihaXcevmZrF1SxvnzFRAiqSKegVED7I163UQQBNG+\nCqwDWoBC4Dp3HzOzWPYFwMxuAG4AWLBgwfRUHquBTvjJJ+GF70L1hfBHd0LFGae3htPkYO8Q9eEd\nTOsb2tl+oBeAnMw06haW8Yk3z2Pt0nLOqy3WIHwiKSrRrYt/CDwPvBFYCjxhZr88kTdw9zuBOwHq\n6upO34BKDc/ADz8KPfvgDbfB6/86pTrhtfYMRgVEhB2tQUDkZaVz0cJSrllVw9olZZxbU0JWRmqf\nTYlIIJ6B0QzMj3pdGy6L9iHgCx6MnLfDzBqBs2LcNzGO6IS3DP78CahNjU547s5Dv23m68/snAiI\n/Kx06haVce2FtaxZUsa5NcWaU1pklopnYDwHnGFmiwkO9tcD7520zR7gcuCXZjYHWA40AJ0x7Hv6\ntTwf3C7bti3lOuFt29/NP/5wCxt2RTivtpi/vfIs1iwpZ2V1kXpPiwgQx8Bw9xEzuxl4nODW2Lvd\nfYuZ3RiuvwP4HHCPmb0IGPApdz8IMNW+8ar1uFK4E17v0AhffmI73/rNLopyMvjXa8/lXRfN15De\nIvIqlkrzKNTV1fnGjRun903bd8LDN0LThpTqhOfu/OiFffyv/9lKa88Q11+8gE/+4XJK809uPg4R\nmZnMbJO718WybaIbvZOXO2z6Fjz+9ynXCW9Hay+fWbeZX+9oZ2VNEd/4QB2r5pckuiwRSXIKjKn0\n7Id1fwWv/DSlOuH1HxrhKz/fwV2/bCA3M53PXbOC965ZSLouP4lIDBQYk219BH50Kwz3w5X/By7+\n8IzvhOfuPL7lAJ/78VaaOwe49sJa/vatZ1FRkJ3o0kRkBlFgjBvsgkc/CS88ANUXwDvvhMozE13V\nKdvd3sdn1m3h6ZfbOGtuId+/8TVcvGjmt8GIyOmnwABo/EXQCa+7Bd7wKXj938z4TniDw6N8/emd\nfP2ZnWSlp/Hpq87mg69dpFtkReSkKTD6I3D/e6Bwbsp0wvv5tgN8dt1W9kT6efv51fz9VWczp0hz\nSojIqVFg5JXB+74P81bN+E54TR39/NOPtvLE1gMsrcznvr9Yw2uXVSS6LBFJEQoMgIWvTXQFp2Ro\nZJS7ftnIV37+CobxqSvO4s8vWawxnkRkWikwZrhfvXKQf3xkMw0H+7hixVz+4epzqCnJTXRZIpKC\nFBgz1P6uQT73P1v5nxf2sbA8j3s+dDGXLq9KdFkiksIUGDPM8OgY9/x6F19+cjvDY87H33Qmf/mG\nJZqkSETiToExg9Q3tPMPj2xm+4Fe3nhWFZ+9egULymd2Q72IzBwKjBmgtWeQ//3oNh7+XTM1Jbnc\n+YGLePM5cwhmthUROT0UGElsZHSM76zfzb/9dDuDI6PcdNlSbr7sDHKzdPlJRE4/BUaS+u2eDj79\n8Ga27uvmkmUV/NM1K1haWZDoskRkFlNgJJlI3yH+9Sfb+O7Gvcwpyuar772Aq86dp8tPIpJwCowk\n8r2Ne/mXR1+iZ3CED//BYm5505kUZOs/kYgkBx2NksTL+3v45IMvcPGiUj7/jnNZPrcw0SWJiBxB\ngZEkfrPzIABfvv4C9dQWkaSkwYaSRH1DhNrSXIWFiCQtBUYScHc27IqwZnF5oksRETkqBUYSeKW1\nl0jfIdYs0Ux4IpK8FBhJoL6hHYC1OsMQkSSmwEgC6xsjzCvOYX6Z2i9EJHkpMBLM3alviLBmcZk6\n54lIUlNgJFjDwT4O9g6xZokuR4lIclNgJFh9QwSANYvV4C0iyU2BkWD1je1UFmazuCI/0aWIiByT\nAiOB1H4hIjOJAiOB9kT62d89qMtRIjIjKDASaKL9Qg3eIjIDxDUwzOwKM3vZzHaY2W1TrP8bM3s+\nfGw2s1EzKwvX7TKzF8N1G+NZZ6Ksb2ynLD+LM6o0MZKIJL+4jVZrZunA7cCbgSbgOTNb5+5bx7dx\n9y8CXwy3vxr4uLtHot7mMnc/GK8aE62+IcLqRWq/EJGZIZ5nGKuBHe7e4O6HgAeAa46x/XuA++NY\nT1Jp6uinuXNA40eJyIwRz8CoAfZGvW4Kl72KmeUBVwA/iFrswJNmtsnMbohblQlyuP+F2i9EZGZI\nlgmUrgZ+Pely1CXu3mxmVcATZrbN3X8xeccwTG4AWLBgwempdhrUN7ZTnJvJWZpZT0RmiJjOMMzs\nITO7ysxO5IykGZgf9bo2XDaV65l0Ocrdm8OfrcDDBJe4XsXd73T3Onevq6ysPIHyEqu+McLFi8pI\nS1P7hYjMDLEGwNeA9wKvmNkXzGx5DPs8B5xhZovNLIsgFNZN3sjMioE3AI9ELcs3s8Lx58BbgM0x\n1pr09ncNsru9n7VqvxCRGSSmS1Lu/iRBe0IxQeP0k2a2F/gm8B13H55inxEzuxl4HEgH7nb3LWZ2\nY7j+jnDTdwI/dfe+qN3nAA+Hdw9lAPe5+2Mn9RsmofrGYP4LtV+IyEwScxuGmZUD7wc+APwOuBe4\nBPhT4NKp9nH3R4FHJy27Y9Lre4B7Ji1rAM6PtbaZZn1DhMLsDM6pLkp0KSIiMYspMMzsYWA58N/A\n1e6+L1z13VTtVBdP9Y3t1C0qJV3tFyIyg8R6hvEf7v7UVCvcvW4a60l5rT2DNLT18e66+cffWEQk\nicTa6H2OmZWMvzCzUjP7aJxqSmkbGjX/hYjMTLEGxofdvXP8hbt3AB+OT0mpbUNjhLysdFbWFCe6\nFBGRExJrYKRb1IBH4ThRWfEpKbXVN0S4aGEpmekaKFhEZpZYj1qPETRwX25mlxN0skuZ21xPl0jf\nIV4+0MNaDWcuIjNQrI3enwL+EvhI+PoJ4K64VJTC1H4hIjNZrB33xoCvhw85SfWN7eRkpnFebcnx\nNxYRSTKx9sM4A/jfwDlAzvhyd18Sp7pSUn1DhAsXlJKVofYLEZl5Yj1yfYvg7GIEuAz4L+A78Soq\nFXX1D/PS/m4NByIiM1asgZHr7j8DzN13u/tngaviV1bqeW5XBHc0YZKIzFixNnoPhUObvxIOKNgM\naCLqE1Df2E5WRhqr5qv9QkRmpljPMG4B8oCPARcRDEL4p/EqKhXVN0ZYNb+EnMz0RJciInJSjhsY\nYSe969y9192b3P1D7n6tu68/DfWlhJ7BYTY3d7FWt9OKyAx23MBw91GCYczlJG3c3cGYwxp12BOR\nGSzWNozfmdk64PvAxERH7v5QXKpKMfUNETLSjAsWqP1CRGauWAMjB2gH3hi1zAEFRgzqG9s5r7aY\nvKyY56so7N1VAAAPhUlEQVQSEUk6sfb0/lC8C0lV/YdGeLGpiw+/Xn0cRWRmi7Wn97cIziiO4O5/\nNu0VpZhNuzsYGXONHyUiM16s10h+HPU8B3gn0DL95aSe+oYI6WlG3SIFhojMbLFekvpB9Gszux/4\nVVwqSjH1je2srC6iIFvtFyIys53sKHhnAFXTWUgqGhwe5fd7u3Q7rYikhFjbMHo4sg1jP8EcGXIM\nv93TwaHRMbVfiEhKiPWSVGG8C0lF9Q0RzFD7hYikhJguSZnZO82sOOp1iZm9I35lpYb6xnbOmVdE\ncW5moksRETllsbZhfMbdu8ZfuHsn8Jn4lJQahkZG+d2eTs1/ISIpI9bAmGo73fZzDL/f28XQyJjm\nvxCRlBFrYGw0s383s6Xh49+BTfEsbKarb2gHYLXaL0QkRcQaGH8FHAK+CzwADAI3xauoVFDfGOGs\nuYWU5mcluhQRkWkR611SfcBtca4lZQyPjrFpdwfvrqtNdCkiItMm1ruknjCzkqjXpWb2ePzKmtle\nbO5iYHhUHfZEJKXEekmqIrwzCgB370A9vY+qviECwGp12BORFBJrYIyZ2YLxF2a2iClGr53MzK4w\ns5fNbIeZveqSlpn9jZk9Hz42m9momZXFsm8yq29sZ1lVARUF2YkuRURk2sR6a+zfA78ys2cAA/4A\nuOFYO4Rzgd8OvBloAp4zs3XuvnV8G3f/IvDFcPurgY+7eySWfZPVyOgYG3d1cM2q6kSXIiIyrWI6\nw3D3x4A64GXgfuATwMBxdlsN7HD3Bnc/RHB31TXH2P494XufzL5JY+u+bnqHRtR+ISIpJ9bBB/8C\nuAWoBZ4H1gLPcuSUrZPVAHujXjcBa47y/nnAFcDNJ7HvDYRnOwsWLJhqk9NqvP1irdovRCTFxNqG\ncQtwMbDb3S8DLgA6j73LCbka+LW7R050R3e/093r3L2usrJyGks6OfWN7SyuyKeqKCfRpYiITKtY\nA2PQ3QcBzCzb3bcBy4+zTzMwP+p1bbhsKtdz+HLUie6bNEbHnA2NEQ1nLiIpKdbAaAr7YfwQeMLM\nHgF2H2ef54AzzGyxmWURhMK6yRuFo+C+AXjkRPdNNtv2d9M9OKLxo0QkJcXa0/ud4dPPmtlTQDHw\n2HH2GTGzm4HHgXTgbnffYmY3huvvCDd9J/DTsDf5Mfc9gd8rIcbbLzRCrYikohMecdbdnzmBbR8F\nHp207I5Jr+8B7oll32RX39jO/LJcqktyE12KiMi0O9k5vWWSsbD9YvUinV2ISGpSYEyTV1p76egf\nVvuFiKQsBcY0qW8M5r9Yq/YLEUlRCoxpUt8QYV5xDvPL1H4hIqlJgTEN3J36xnbWLC7DzBJdjohI\nXCgwpsHOtj4O9h7S+FEiktIUGNNgvP1CPbxFJJUpMKZBfUOEysJsFlfkJ7oUEZG4UWCcIrVfiMhs\nocA4Rbvb+znQPaT2CxFJeQqMU3S4/4XaL0QktSkwTlF9Q4Ty/CyWVRUkuhQRkbhSYJyi+sYIq9V+\nISKzgALjFOyN9NPcOaDbaUVkVlBgnIINjeH8F2rwFpFZQIFxCuob2ynJy2T5nMJElyIiEncKjFNQ\n3xjh4kVlpKWp/UJEUp8C4yTt7xpkd3u/2i9EZNZQYJykif4Xar8QkVlCgXGS1jdEKMzJ4Ox5RYku\nRUTktFBgnKT6xnYuXlRGutovRGSWUGCchNaeQRra+tR+ISKzigLjJKj/hYjMRgqMk1DfECE/K52V\n1Wq/EJHZQ4FxEuob27loURkZ6fr6RGT20BHvBEX6DrH9QK/aL0Rk1lFgnKANmr9bRGYpBcYJWt8Q\nISczjfNqSxJdiojIaaXAOEH1jREuXFBKVoa+OhGZXXTUOwFd/cNs29/NmsW6nVZEZh8FxgnYsCuC\nO6xZovYLEZl94hoYZnaFmb1sZjvM7LajbHOpmT1vZlvM7Jmo5bvM7MVw3cZ41hmr+oZ2sjLSWDVf\n7RciMvtkxOuNzSwduB14M9AEPGdm69x9a9Q2JcDXgCvcfY+ZVU16m8vc/WC8ajxR9Y0RVs0vIScz\nPdGliIicdvE8w1gN7HD3Bnc/BDwAXDNpm/cCD7n7HgB3b41jPaeke3CYLS1drNXttCIyS8UzMGqA\nvVGvm8Jl0c4ESs3saTPbZGZ/ErXOgSfD5Tcc7UPM7AYz22hmG9va2qat+Mk27epgzDV+lIjMXnG7\nJHUCn38RcDmQCzxrZuvdfTtwibs3h5epnjCzbe7+i8lv4O53AncC1NXVebwKXd/YTma6ceGC0nh9\nhIhIUovnGUYzMD/qdW24LFoT8Li794VtFb8Azgdw9+bwZyvwMMElroSpb4hwXm0JuVlqvxCR2Sme\ngfEccIaZLTazLOB6YN2kbR4BLjGzDDPLA9YAL5lZvpkVAphZPvAWYHMcaz2mvqERXmzu0nAgIjKr\nxe2SlLuPmNnNwONAOnC3u28xsxvD9Xe4+0tm9hjwAjAG3OXum81sCfCwmY3XeJ+7PxavWo9n0+4O\nRsdc7RciMqvFtQ3D3R8FHp207I5Jr78IfHHSsgbCS1PJoL6xnfQ046KFar8QkdlLPb1jsKExwsqa\nYgqyE32PgIhI4igwjmNweJTf71X/CxERBcZx/HZPB4dGxzR+lIjMegqM46hviJBmULdIgSEis5sC\n4zjqG9s5p7qIopzMRJciIpJQCoxjGBoZ5Xd7OjX/hYgICoxj+v3eLoZGxtRhT0QEBcYx1Te0Ywar\nFRgiIgqMY6lvjLB8TiEleVmJLkVEJOEUGEcxPDrGpt0drNVwICIigALjqF5o6mJgeFTtFyIiIQXG\nUdQ3tgNwsQJDRARQYBxVfUOEZVUFVBRkJ7oUEZGkoMCYwsjoGBt3RXQ5SkQkigJjCltauuk7NKr5\nL0REoigwpjDefqERakVEDlNgTKG+IcLiinyqinISXYqISNJQYEwyOuZsUPuFiMirKDAmeWlfNz2D\nI5r/QkRkEgXGJPWNEQCNUCsiMokCY5L6hnbml+VSXZKb6FJERJKKAiPK2ET7hc4uREQmU2BE2d7a\nQ2f/sBq8RUSmoMCIUt8QtF9ohFoRkVdTYESpb2ynujiH2lK1X4iITKbACLk7GxojrFlSjpkluhwR\nkaSjwAjtbOvjYO8htV+IiByFAiM0Pn6UBhwUEZmaAiNU3xChqjCbReV5iS5FRCQpKTAI2i/qG9vV\nfiEicgwKDGB3ez8HuofUfiEicgxxDQwzu8LMXjazHWZ221G2udTMnjezLWb2zInsO10m5r/QgIMi\nIkeVEa83NrN04HbgzUAT8JyZrXP3rVHblABfA65w9z1mVhXrvtOpviFCRUEWSysL4vH2IiIpIZ5n\nGKuBHe7e4O6HgAeAayZt817gIXffA+DurSew77Spb4ywenGZ2i9ERI4hnoFRA+yNet0ULot2JlBq\nZk+b2SYz+5MT2BcAM7vBzDaa2ca2trYTLnJweJTXLi3nD1fMPeF9RURmk7hdkjqBz78IuBzIBZ41\ns/Un8gbufidwJ0BdXZ2faAE5mel88V3nn+huIiKzTjwDoxmYH/W6NlwWrQlod/c+oM/MfgGcHy4/\n3r4iInIaxfOS1HPAGWa22MyygOuBdZO2eQS4xMwyzCwPWAO8FOO+IiJyGsXtDMPdR8zsZuBxIB24\n2923mNmN4fo73P0lM3sMeAEYA+5y980AU+0br1pFROT4zP2EL/snrbq6Ot+4cWOiyxARmTHMbJO7\n18WyrXp6i4hITBQYIiISEwWGiIjERIEhIiIxSalGbzNrA3af5O4VwMFpLGcm03dxJH0fR9L3cVgq\nfBcL3b0ylg1TKjBOhZltjPVOgVSn7+JI+j6OpO/jsNn2XeiSlIiIxESBISIiMVFgHHZnogtIIvou\njqTv40j6Pg6bVd+F2jBERCQmOsMQEZGYKDBERCQmsz4wzOwKM3vZzHaY2W2JrieRzGy+mT1lZlvN\nbIuZ3ZLomhLNzNLN7Hdm9uNE15JoZlZiZg+a2TYze8nMXpPomhLJzD4e/jvZbGb3m1lOomuKt1kd\nGGaWDtwOXAmcA7zHzM5JbFUJNQJ8wt3PAdYCN83y7wPgFoI5WgT+H/CYu59FMNHZrP1ezKwG+BhQ\n5+4rCaZhuD6xVcXfrA4MYDWww90b3P0Q8ABwTYJrShh33+fuvw2f9xAcEKacS302MLNa4CrgrkTX\nkmhmVgy8HvhPAHc/5O6dia0q4TKAXDPLAPKAlgTXE3ezPTBqgL1Rr5uYxQfIaGa2CLgAqE9sJQn1\nZeCTBJN7zXaLgTbgW+ElurvMLD/RRSWKuzcDXwL2APuALnf/aWKrir/ZHhgyBTMrAH4A3Oru3Ymu\nJxHM7G1Aq7tvSnQtSSIDuBD4urtfAPQBs7bNz8xKCa5GLAaqgXwze39iq4q/2R4YzcD8qNe14bJZ\ny8wyCcLiXnd/KNH1JNDrgLeb2S6CS5VvNLPvJLakhGoCmtx9/IzzQYIAma3eBDS6e5u7DwMPAa9N\ncE1xN9sD4zngDDNbbGZZBI1W6xJcU8KYmRFco37J3f890fUkkrv/rbvXuvsigv8vfu7uKf8X5NG4\n+35gr5ktDxddDmxNYEmJtgdYa2Z54b+by5kFNwFkJLqARHL3ETO7GXic4C6Hu919S4LLSqTXAR8A\nXjSz58Nlf+fujyawJkkefwXcG/5x1QB8KMH1JIy715vZg8BvCe4u/B2zYJgQDQ0iIiIxme2XpERE\nJEYKDBERiYkCQ0REYqLAEBGRmCgwREQkJgoMkSRgZpdqRFxJdgoMERGJiQJD5ASY2fvNbIOZPW9m\n3wjny+g1s/8bzo3wMzOrDLddZWbrzewFM3s4HH8IM1tmZk+a2e/N7LdmtjR8+4Ko+SbuDXsQiyQN\nBYZIjMzsbOA64HXuvgoYBd4H5AMb3X0F8AzwmXCX/wI+5e7nAS9GLb8XuN3dzycYf2hfuPwC4FaC\nuVmWEPS8F0kas3poEJETdDlwEfBc+Md/LtBKMPz5d8NtvgM8FM4fUeLuz4TLvw1838wKgRp3fxjA\n3QcBwvfb4O5N4evngUXAr+L/a4nERoEhEjsDvu3uf3vEQrN/mLTdyY63MxT1fBT9+5Qko0tSIrH7\nGfDHZlYFYGZlZraQ4N/RH4fbvBf4lbt3AR1m9gfh8g8Az4QzGTaZ2TvC98g2s7zT+luInCT9BSMS\nI3ffamafBn5qZmnAMHATwWRCq8N1rQTtHAB/CtwRBkL06K4fAL5hZv8cvse7TuOvIXLSNFqtyCky\ns153L0h0HSLxpktSIiISE51hiIhITHSGISIiMVFgiIhITBQYIiISEwWGiIjERIEhIiIx+f9tEYch\nv1PJAwAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x19347f78a90>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.plot(history.history['acc'])\n",
    "plt.plot(history.history['val_acc'])\n",
    "plt.title('model accuracy')\n",
    "plt.ylabel('accuracy')\n",
    "plt.xlabel('epoch')\n",
    "plt.legend(['train', 'val'], loc='upper left')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.1"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
